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Abstract 

Collective  command  and  control  (C2)  civil-military  operations  impose  a  strong  requirement 
for  organizations  and  people  to  work  collaboratively  and  to  interact  with  information  more 
efficiently.  Human-Computer  Interaction  (HCI)  technology  has  been  evolving  at  a  great 
pace  during  the  last  decade  and  many  novel  HCI  concepts  and  tools  have  become  of  interest 
to  support  not  only  analysts  deployed  in  command  centers  but  also  warfighters  deployed  in 
the  field.  This  will  have  an  impact  on  how  intelligence  officers  will  operate  in  the  future.  In 
this  regard,  Defence  Research  and  Development  (R&D)  (DRDC)  Canada  has  undertaken  a 
major  initiative  to  look  forward  into  Future  Intelligence  Analysis  Capabilities  (FIAC).  This 
paper  reports  on  HCI  trends  and  applicability  to  support  the  FIAC  and  collective  C2  in 
general.  Topics  of  interest  include  smart-room  and  collaborative  working  environments, 
hardware  display  technology  (e.g.  large  group  displays,  flexible  displays,  wearable 
displays),  mixed/augmented  reality,  mobile  and  ubiquitous  computing,  multi-modal 
interaction  (e.g.  touch  tables,  speech  and  gesture  recognition),  biometry,  intelligent  and 
adaptive  user  interfaces,  and  advanced  information  visualization  that  allows  to  analyze  and 
comprehend  multi-dimensional  and  complex  information. 

1  Introduction 

The  level  of  operational  complexity  facing  (military)  intelligence  has  hugely  increased  in 
recent  times  (DLCD,  2009).  Recent  confrontations  have  been  largely  characterized  by 
(Poussart,  2011): 

•  a  prevalence  of  asymmetric  warfare  such  as  in  urban  environments  or  when  dealing 
with  rogue  states  or  terrorism; 

•  the  emergence  of  cyber  warfare; 

•  the  emergence  of  instantaneous  connections  between  the  tactical  and  the  strategic 
levels,  and  the  increased  linkages  between  the  local,  regional  and  global  spaces; 

•  the  complexity  and  diversification  of  human  terrains  and  contexts; 

•  the  conduct  of  Joint,  Interagency,  Multinational  and  Public  (JIMP)  operations;  and 

•  the  continued  progression  of  Communication  and  Information  Technology,  which 
provides  significant  opportunities  to  better  conduct  military  operations. 

In  order  to  address  the  new  context  of  military  operations,  from  an  intelligence  perspective, 
DRDC  is  conducting  the  FIAC  project.  The  core  objective  of  the  FIAC  is  to  advance  the 
state-of-the-art  of  intelligence  methods  and  resources  in  order  to  adapt  to  and  anticipate  the 


new  challenges  of  the  Canadian  Forces  (CF).  FIAC  addresses  information  services  from  an 
all  sources  perspective,  over  the  full  intelligence  cycle  but  with  particular  emphasis  on 
advanced  analysis  capabilities.  It  seeks,  in  particular,  to  achieve  synergy  between  human 
cognition  and  machine  intelligence,  fully  exploiting  collaborative  interaction.  FIAC  seeks  to 
address  the  demands  of  intelligence  in  the  modem  operating  terrain  and  the  expanding 
uncertainties  that  the  CF  are  facing. 

Over  the  last  year,  a  number  of  initiatives  have  been  conducted  to  set  the  stage  for  FIAC. 
This  includes  the  development  of  a  white  paper  (Poussart,  2011),  the  production  of  an 
illustration  of  a  future  multi-intelligence  center  (Figure  1),  the  development  of  a  fiction 
narrative  of  a  future  intelligence  environment,  the  development  of  Operating  concepts  and  a 
state  of  the  art  review  of  technological  trends  in  terms  of  HCI  and  intelligence  analysis 
capabilities.  This  paper  reports  on  the  advances  in  HCI  that  could  be  exploited  in  future 
intelligence  environments  and  FIAC. 


Figure  1-  Illustration  of  a  future  multi-intelligence  center  (Poussart,  2011) 

2  Future  Intelligence  Context 

The  purpose  of  intelligence  is  to  provide  commanders  and  staffs  with  timely,  relevant, 
accurate,  predictive,  and  tailored  information  about  the  enemy  and  other  aspects  of  the  area 
of  operations.  Intelligence  supports  the  planning,  preparation,  execution,  and  assessment  of 
operations  (Department  of  the  Army,  2010). 

Military  analysts  need  to  assess,  understand  and  predict  the  development  of  a  situation.  To 
this  end,  they  need  to  handle  a  continuously  growing  set  of  information  of  different 
primitive  data  types  and  of  different  sources.  Sources  include:  communication  and  electronic 
signal  information,  geospatial  intelligence,  imagery,  meteorological  and  oceanographic 
information,  human  intelligence,  open-source  intelligence,  as  well  as  information  from  other 
government  departments  and  allied  countries. 

With  an  ever-expanding  set  of  sensing  modalities  and  sophisticated  sensing  platforms,  and 
this  increased  volume  of  data  and  information  that  they  need  to  collect  and  process,  analysts 
are  faced  with  data  and  cognitive  overload.  There  is  a  need  for  more  efficient  information 
management  and  analysis  capabilities,  as  well  as  better  HCI  capabilities  to  support 
collaboration  and  interaction  with  information.  These  enhanced  capabilities  must  be 
provided  both  for  analysts  in  command  centers  as  well  as  for  deployed  collators  or  analysts. 

From  an  HCI  perspective,  future  intelligence  environments  should  provide  the  following 
capabilities: 


•  Support  productive  collaboration  among  analysts  with  various  backgrounds  and 
expertise  (multi-intelligence); 

•  Support  collaboration  under  a  JIMP  framework;  and, 

•  Support  multiple  parallel  theatres  of  operations  and  deal  with  several  possibly  but 
not  necessarily  intelligence  requests. 


3  Hype  Cycle  for  Human-Computer  Interaction 

The  design  of  HCI  should  take  into  account  three  interaction  aspects:  physical,  cognitive, 
and  affective.  “The  physical  aspect  determines  the  mechanics  of  interaction  between  human 
and  computer  while  the  cognitive  aspect  deals  with  ways  that  users  can  understand  the 
system  and  interact  with  it.  The  affective  aspect  is  a  more  recent  issue  and  it  tries  not  only  to 
make  the  interaction  a  pleasurable  experience  for  the  user  but  also  to  affect  the  user  in  a  way 
that  make  user  continue  to  use  the  machine  by  changing  attitudes  and  emotions  toward  the 
user”  (Karray  et  al.,  2008). 

The  field  of  human-machine  interface  continues  to  go  through  rapid  changes  with  the 
introduction  of  new  multi-sensory  interfaces  (speech,  sound,  haptics)  and  metaphors 
(gestures,  avatar  in  augmented  or  virtual  reality  world,  shared  cognitive  spaces).  Large 
interactive  displays,  smart  devices  and  embedded  systems  become  more  and  more  pervasive. 

Figure  2  shows  the  status  of  HCI  technology  on  the  hype  cycle,  a  graphic  representation  of 
the  maturity  and  adoption  of  technologies  and  applications,  showing  emerging  technologies 
as  well  as  indicating  which  technologies  have  gone  through  convincing  focused 
experimentation  (slope  of  enlightment)  and  have  found  adoption  (plateau  of  productivity) 
(Gartner,  2010). 
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Figure  2  -  Hype  Cycle  for  Human-Computer  Interaction,  2010  (Gartner,  2010) 

The  next  sections  examine  a  number  of  the  most  applicable  or  promising  HCI  technologies 
to  support  the  military  intelligence  community. 


4  Ubiquitous  and  Embedded  Computing 

Ubiquitous  computing  has  been  named  the  Third  Wave  of  computing.  The  First  Wave  was 
the  mainframe  era,  many  people  one  computer.  Then  it  was  the  Second  Wave,  one  person 
one  computer  which  was  called  Personal  Computer  era  and  now,  Ubiquitous  computing 
introduces  the  many  computers  one  person  era  (Riva,  2005).  People  are  able  to  interact  with 
the  technology  that  surrounds  them  in  more  accessible,  intuitive  and  less  restrictive  ways. 

In  a  growing  ubiquitous  computing  world,  computers  will  communicate  through  high  speed 
local  networks,  over  wide-area  networks,  and  via  infrared,  ultrasonic,  cellular,  and  other 
technologies.  Data  and  computational  services  will  be  portably  accessible  from  many,  if  not 
most,  locations  to  which  a  user  travels.  Computation  will  pass  beyond  desktop  computers 
into  every  object  for  which  uses  can  be  found.  The  environment  will  be  alive  and  the 
addition  of  networked  communications  will  allow  many  of  these  embedded  computations  to 
coordinate  with  each  other  and  with  the  user. 

Closely  related  to  ubiquitous  computing,  location-aware  technology  includes  sensors  and 
methods  for  detecting  or  calculating  the  geographical  position  of  a  person,  a  mobile  device 
or  other  moving  objects.  The  most  common  location-aware  technologies  are  Global 
Positioning  System  (GPS),  Assisted  GPS  (A-GPS),  Enhanced  Observed  Time  Difference 
(EOTD)  and  Enhanced  GPS  (E-GPS).  High-accuracy  location  technologies  for  private 
Wireless-Fidelity  (Wi-Fi)  networks  also  provide  location  services  within  buildings  and  hot 
spots  (Gartner,  2010).  Location-aware  technology  is  key  for  situation  awareness  in  the 
conduct  of  military  operations. 

5  Multimodal  and  Natural  Interaction 

“A  multimodal  interface  acts  as  a  facilitator  of  human-computer  interaction  via  two  or  more 
modes  of  input  that  go  beyond  the  traditional  keyboard  and  mouse.  The  exact  number  of 
supported  input  modes,  their  types  and  the  way  in  which  they  work  together  may  vary 
widely  from  one  multimodal  system  to  another.  Multimodal  interfaces  incorporate  different 
combinations  of  speech,  gesture,  gaze,  facial  expressions  and  other  non-conventional  modes 
of  input”  (Oviatt,  2003). 

5.1  Gesture  Recognition 

Gesture  recognition  involves  determining  the  movement  of  a  user’s  fingers,  hands,  arms, 
head  or  body  in  three  dimensions  through  the  use  of  a  camera;  or  via  a  device  with 
embedded  sensors  that  may  be  worn,  held  or  body-mounted  (Gartner,  2010). 

The  primary  application  for  gestural  interfaces  at  present  is  in  the  gaming  and  home 
entertainment  market.  However,  for  the  military,  the  potential  of  hands-free  control  of 
devices,  and  the  ability  for  several  people  to  interact  with  large  datasets,  opens  up  a  wide 
range  of  business  applications,  including  data  visualization  and  analytics,  and  the  interaction 
with  Large  Group  Displays  (LGDs). 

5.1.1  Handheld  controllers 

The  commercialization  of  gesture  interfaces  began  with  handheld  devices  that  detect  motion, 
such  as  the  Nintendo  Wii’s  three-dimensional  (3D)  controller  (Figure  3a),  3D  mice  and 


high-end  mobile  phones  with  accelerometers.  Sony  has  also  revealed  its  alternative,  called 
Move,  which  uses  a  handheld  controller  similar  to  the  Nintendo  Wii  device,  but  claims 
greater  resolution. 


WnMote 


Figure  3a  -  WiiMote;  Figure  3b  -  Xbox  Kinect  controllers 
5.1.2  Camera-based  recognition 

Camera-based  systems  are  now  entering  the  market.  One  of  the  most  visible  examples  is  the 
newly  launched  Microsoft  Kinect  gaming  controller  for  the  Xbox  360  (Figure  3b).  Using  a 
webcam-style  peripheral,  it  allows  users  to  control  and  interact  with  the  Xbox  360  through  a 
natural  user  interface  using  gestures  and  spoken  commands,  and  without  the  need  to  touch  a 
game  controller.  Although  it  was  intended  for  game  play,  developers  saw  the  huge  potential 
this  low  cost  sensor  could  have  for  computer  interaction  and  an  open  source  PC  driver  was 
developed  by  hackers  within  days  (hours).  Microsoft  announced  they  will  support  PC  use  of 
the  Kinect  motion  controller  (BBC,  2011). 

Kinect  is  capable  of  simultaneously  tracking  up  to  six  people,  including  two  active  players 
for  motion  analysis  with  a  feature  extraction  of  20  joints  per  player,  focused  on  the  head, 
hands,  feet  and  torso.  However,  Kinect  is  capable  of  tracking  more  points  than  those  20 
points.  For  instance,  Massachusetts  Institute  of  Technology’s  (MIT’s)  Robot  Locomotion 
Group  and  Learning  Intelligent  Systems  Teams  have  shown  that  the  Kinect  can  recognize 
ten  fingers  and  some  relatively  small  gestures,  similar  to  the  interaction  shown  in  the  movie 
Minority  Report  (Nosowitz,  2010). 

A  spin-off  from  the  Minority  Report  movie,  the  G-Speak  spatial  computing  operating 
system  stands  at  the  higher  end  of  the  market.  It  was  developed  by  John  Underkoffler  at 
MIT  Media  Lab  and  now  commercialized  by  Oblong  Industries  Inc.  (Oblong,  2011).  In 
addition  to  the  camera-based  gesture  interaction  (Figure  4),  this  system  offers  a  management 
capability  that  allows  data  to  be  moved  between  different  computing  systems  and  displays. 


Figure  4-  G-Speak  gesture  interaction  (Oblong,  2011) 


5.2  Multitouch 


“Multitouch  refers  to  a  touchscreen  interaction  technique  in  which  multiple  simultaneous 
touchpoints  and  movements  can  be  detected  and  used  to  control  objects  in  a  user  interface  or 
other  application.  A  user  may,  for  example,  zoom  into  a  picture  or  Web  page  by  placing 
thumb  and  index  finger  on  a  touchscreen  and  then  moving  them  apart.  To  zoom  back  out, 
the  user  would  then  move  the  same  two  fingers  back  together... A  multitouch  user  interface 
is  becoming  a  must-have  feature  of  smartphones,  media  tablets  and  portable  consumer 
electronics  devices...  Multitouch  will  change  use  models  and  will  have  a  significant  impact 
on  product  design”  (Gartner,  2010). 

5.3  Multitouch  Tables  /  Surface  Computing 

“Surface  computers  are  large-screen  displays  that  support  direct  interaction  via  touch  or 
gesture  (as  opposed  to  external  devices,  such  as  mice  or  keyboards).  They  are  typically 
horizontal,  often  built  into  the  furniture,  such  as  a  table  top,  but  may  be  delivered  as  vertical 
wall-mounted  or  free-standing  displays.  The  displays  incorporate  much  of  the  style  of 
interaction  (such  as  rotate,  pinch,  zoom  and  flick  movements)  found  in  multitouch  devices 
but  can  typically  recognize  more  than  one  set  of  touches  at  a  time,  enabling  multiple  users  to 
interact  or  work  collaboratively.  Some  also  have  the  capability  to  recognize  physical  objects 
marked  with  a  special  identification  tag,  allowing  context-sensitive  information  to  be 
provided  when  items  are  placed  on  the  display.  Their  size  is  constrained  by  the  ability  to 
physically  reach  across  the  surface.  Larger  displays  may  require  a  noncontact  approach 
involving  a  gestural  interface,  where  the  user  does  not  need  to  physically  touch  the  surface” 
(Gartner,  2010). 

In  a  military  intelligence  context  multitouch  table  will  likely  be  very  valuable  allowing 
various  analysts  to  come  together  around  the  table  to  debate  on  a  military  situation. 

5.4  Speech  Recognition  and  Translation 

Speech  recognition  converts  spoken  words  to  machine-readable  input  while  voice 
recognition  is  a  system  trained  to  a  particular  user,  which  recognizes  their  speech  based  on 
their  unique  vocal  sound.  Transcription  is  the  conversion  of  a  spoken-language  source  into 
written  /  typewritten.  Automatic  translation  is  the  conversion  of  a  spoken-language  source 
into  another  language  and  is  still  a  very  challenging  task  in  terms  of  recognition  of  language 
structures. 

Speech  recognition  can  have  large  applicability  in  a  military  intelligence  setting  where  input 
from  deployed  observers  and  collators  are  digitized  or  as  part  of  collaboration  activities 
where  automated  transcripts  of  discussions  are  produced.  Considering  that  intelligence 
analysts  need  to  digest  messages  and  documents  written  in  various  languages,  automatic 
translation  has  also  a  significant  applicability.  It  also  has  applicability  to  the  deployed 
collators  interacting  with  local  populations,  using  low-costs  apps  developed  for  the  smart 
phones. 

5.5  Emotion  Recognition  Multimodal  Systems 

People  are  able  to  perceive  one’s  emotional  state  based  on  their  observations  about  one’s 
face,  body,  and  voice.  Research  in  multimodal  systems  have  been  conducted  to  allow  the 


inferring  of  one’s  emotional  state  based  on  various  behavioral  signals.  A  bimodal  system 
based  on  fusing  the  facial  recognition  and  acoustic  information,  provided  an  accurate 
classification  of  89.1  percent  in  terms  of  emotion  recognition  of  ‘sadness,  anger,  happiness, 
and  neutral  state’  (Karray  et  al.,  2008). 

In  a  military  intelligence  context,  emotion  recognition  may  provide  valuable  clues  about  a 
person’s  stress  level,  intent,  or  trust  level.  This  may  be  very  important  in  a  JIMP  context 
where  interaction  takes  place  with  people  from  various  organizations,  often  through  audio  / 
video  collaboration  or  in  a  context  where  interaction  takes  place  with  elusive  and  changing 
adversary. 

6  Display  Technology 

6.1  Large  Group  Displays 

For  decades,  command  centers  have  been  equipped  with  LGDs.  However,  these 
technologies  are  not  always  deployed  and  used  effectively.  In  many  cases,  the  usage 
contexts  are  not  well  thought  through.  Gouin  et  al.  (2009)  provide  a  number  of  human  factor 
guidelines  for  the  use  of  LGDs,  including  human  perception/legibility,  information 
organization  and  display  control. 

Technology  for  LGDs  is  continuously  evolving,  resulting  in  the  availability  of  larger  and 
more  capable  displays  at  lower  costs  as  well  as  new  interaction  and  collaboration 
technologies,  such  as  the  use  of  multi-modal  interaction  and  more  comprehensive  support 
for  co-located  and  distributed  collaboration.  Figures  5a  and  5b  show  respectively  a  rear- 
projection  display  and  a  mosaic  of  LCD  panels. 


Figure  5a  -  BARCO  OV-1015 100” SXGA+DLP Projection  Module  (Barco,  2011); 

Figure  5b  -  Sharp  PN-V601  60  Inch  LCD  Video  Wall  (Isignpak,  2011) 

The  Interactive  DataWall  (Figure  6)  developed  at  the  Air  Force  Research  Laboratory 
(AFRL)  is  a  good  example  of  how  multi-modal  interaction  can  apply  to  LGDs.  It  is  built 
using  three  horizontally  tiled  video  projectors  each  displaying  1280  x  1024  pixels  for  a 
combined  resolution  of  3840  x  1024  pixels  across  a  12'  x  3'  screen  area.  The  system  also 
features  speaker-independent  voice  activation  and  a  wireless  pointing  device  using  camera 
tracked  laser  pointers  (AFRL,  2001). 


Figure  6  -  AFRL  Interactive  Data  Wall  (AFRL,  2001) 

6.2  OLED  Displays 

Organic  Light  Emitting  Diodes  (OLEDs)  are  a  flat  display  technology,  made  by  placing  a 
series  of  organic  thin  films  between  two  conductors.  OLEDs  are  called  organic  because  they 
are  made  from  carbon  and  hydrogen.  When  electrical  current  is  applied,  a  bright  light  is 
emitted.  Because  OLEDs  produce  (emit)  light  they  do  not  require  a  backlight  (OLED  Info, 
2011).  Some  key  advantages  for  the  military  of  OLEDs  over  today’s  flat-panel  technology 
(Liquid  Crystal  Dsiplay  (LCD)  or  plasma)  are  that  they  can  be  ultra-thin,  flexible  (Figure  7a) 
and  transparent  (Figure  7b),  have  low  power  consumption,  a  greater  brightness,  a  fuller 
viewing  angle  and  can  operate  in  a  broader  temperature  range.  This  provides  the  potential 
for  curved  OLED  displays,  placed  on  non-flat  surfaces;  wearable  OLEDs;  and  transparent 
OLEDs  embedded  in  windows  or  truck  windshields. 


Figure  7a  -  OLED  flexible  displays  (Pink  Tentacle,  2011); 

7b  -  Samsung  transparent  OLED  display  (CES,  2009) 

6.3  Head-Mounted  Displays  /  Retinal  Displays 

“Head-mounted  displays  (HMDs)  are  small  displays  or  projection  technology  integrated  into 
eyeglasses  or  mounted  onto  a  helmet  or  hat  (Figure  8).  Heads-up  displays  are  a  type  of 
HMD  that  do  not  block  the  user’s  vision,  but  superimpose  the  image  on  the  user’s  view  of 
the  real  world.  An  emerging  form  of  heads-up  display  is  a  retinal  display  that  ‘paints’  a 
picture  directly  on  the  sensitive  part  of  the  user’s  retina.  Although  the  image  appears  to  be 
on  a  screen  at  the  user’s  ideal  viewing  distance,  there  is  no  actual  screen  in  front  of  the  user, 
just  special  optics  (for  example,  modified  eyeglasses)  that  reflect  the  image  back  into  the 
eye.  Some  HMDs  incorporate  inertial  sensors  to  determine  direction  and  movement  (for 
example,  to  provide  context-sensitive  geographic  information)  or  as  the  interface  to  an 
immersive  virtual  reality  application”  (Gartner,  2010). 


Figure  8  -  Head  Mounted  Displays  (Nowak,  2007);  (Park  Service,  2011) 


6.4  Pico  Projectors 

“Pico  projectors  are  very  small  projector  modules  which  can  be  integrated  into  mobile 
devices  such  as  handsets  or  laptops,  or  used  to  create  highly  portable  projector  accessories 
for  mobile  workers”  (Gartner,  2010).  An  example  is  shown  in  Figure  9. 


Figure  9  -  Pico  Projector  (Microvision,  2011) 


6.5  Advanced  Workstations 

Intelligent  analysts  need  workstations  where  they  can  organize  and  interact  with  information 
efficiently.  Large  screen  real-estate  is  necessary  to  display  various  information  elements 
together.  Vertical  and  horizontal  planes  are  considered.  Following  are  examples  of 
interesting  workstations. 

Zenview  is  commercializing  a  multi  monitor  workstation  (Figure  10a).  National  Information 
and  Communications  Technology  Australia  (NICTA)  has  developed  a  collaborative 
workstation,  known  as  Braccetto  (Figure  10b). 


Figure  10a-  Zenview  Multi  monitor  display  (High  Impact  PC,  2011); 

Fig  10b  -  Braccetto  collaboration  workstation  (NICTA,  2011) 

The  Benddesk  workstation  (Figure  11)  is  a  prototype  workstation  combining  a  vertical 
display  together  with  a  horizontal  display,  allowing  continuous  multitouch  interaction 
between  both  areas  (Edwards,  2010). 


Figure  11  -  Benddesk  prototype  workstation 


7  Smart  Phones 


Over  the  last  two  years,  smart  phone  technology,  has  gone  through  significant  evolution. 
“Multitouch,  inertial  sensors,  accelerometers,  location  awareness,  video  analysis  and  even 
direction  are  all  becoming  standard  functionalities  for  high-end  smartphones,  enhancing 
familiarity  with  these  nontraditional  sensory  interfaces  and  encouraging  the  move  toward 
useful  augmented  reality  applications”  (Gartner,  2010).  See  Figure  12  for  examples. 


In  the  United  States,  smart  phones  are  being  adopted  to  support  military  operations.  A 
brigade  at  Fort  Bliss,  Texas  is  being  modernized  through  a  range  of  electronic  devices, 
including  smartphones,  tablet  devices,  e-reader  and  mini-projectors.  Dedicated  military 
applications  will  be  developed,  including  one  that  allows  soldiers  to  track  colleague’s 
locations  on  the  battlefield.  Special  card  readers  will  allow  secure  access  to  data  (Mobile, 
2010). 


Figure  12  -  Smart  Phones  with  Augmented  Reality  ( Zonkio ,  2009)  and  (Moor,  2009) 


One  can  think  that  a  number  of  applications  will  be  available  for  deployed  intelligence 
collators  and  analysts,  such  as:  conferencing,  culturally-assisted  translation,  information 
aggregator,  live  status  tracking,  dynamic  route  finder  (taking  into  account  mission  objectives 
and  threats),  biometry-based  (e.g.  facial)  recognition,  virtual  assistant. 


8  Collaboration  Technologies 

In  a  JIMP  context,  military  operations  are  increasingly  dynamic  and  complex  and  require 
significant  collaboration  and  coordination  between  people  from  various  military  and  civilian 
organizations. 

8.1  Computer  Supported  Cooperative  Work 

Over  the  last  several  years,  a  diversity  of  tools  has  proliferated  to  support  collaboration 
across  distributed  teams.  This  includes  technologies  such  as  audio  /  video  conferencing,  chat 


/  instant  messaging,  electronic  white  board,  application  sharing,  groupware,  collaborative 
real-time  editors,  wikis,  internet  forums,  blogs  and  microblogs,  social  network  services  (e.g. 
Facebook,  Twitter),  virtual  worlds  and  crowd  sourcing.  Allied  organizations  are  using  these 
tools  but  belligerents  are  using  them  as  well.  Crowd  sourcing  has  become  a  significant 
source  of  information  when  an  emergency  situation  occurs  and  people  report  on  it  using  cell 
phones  video  feeds  and/or  text  messages. 

8.2  Pervasive  Human  Sensing  and  Remote  Assistance 

Deployed  military  and  civilian  personnel  could  provide  live  data  streams,  using 
instrumentation  such  as  head-mounted  cameras,  to  a  multi-intelligence  center.  These 
deployed  personnel  could  also  benefit  from  virtual  assistance.  Using  HMDs  collaboration 
could  take  place  between  deployed  soldiers  and  a  military  specialist  in  a  command  centre. 
For  example,  using  a  bidirectional  feed,  the  helpful  advices  of  a  translator,  cultural  expert, 
explosive  device  specialist,  intelligence  analyst  or  medical  staff  could  be  provided  directly 
to  a  soldier  in  the  field.  Theses  specialists  could  insert  information  directly  into  the  HMD  of 
the  soldier.  Local  population  could  help  soldiers  on  cultural  and  ethnic  issues  directly  from  a 
local  command  center  and  not  be  exposed  as  helping  the  forces. 

8.3  Telepresence  /  Teleimmersion 

Telepresence  technology  allows  people  to  interact  as  if  they  were  physically  together  in  the 
same  room.  In  telepresence  videoconferencing,  a  higher  level  of  fidelity  is  provided  in  terms 
of  image  and  audio  quality.  Figure  13  is  an  example  of  Digital  Video  Enterprises  (DVE) 
Telepresence  system  (DVE,  2011). 


Figure  13  -  DVE  Telepresence  system  (DVE,  2011) 


In  a  future  telepresence  setting,  we  could  think  that  intelligence  analysts  will  also  be  able  to 
exchange  documents  just  by  moving  them  to  the  other  participants,  as  if  they  were  in  the 
same  room. 

8.4  LiveSpaces 

The  Defence  Science  &  Technology  Organization  (DSTO)  in  Australia  has  developed  an 
environment  called  LiveSpaces,  seamlessly  integrating  various  technologies,  to  support  both 
co-located  and  distributed  teams  engaged  in  intense  collaborative  activities  such  as  analysis, 
planning  and  decision  making  (Phillips,  2008).  These  types  of  tasks  have  high  cognitive 
requirements  and  cognitive  abilities  which  can  be  easily  overloaded  when  too  much 
attention  needs  to  be  applied  to  simply  managing  and  interacting  with  the  environment,  the 
devices,  applications  and  required  information. 


Figure  14  shows  an  example  of  a  LiveSpaces  environment,  where  users  have  access  to  a 
range  of  shared  displays  including  LGDs  and  a  number  of  smaller  displays  used  to  provide 
ambient  information  and  for  activities  such  as  video  teleconferencing.  The  underlying 
LiveSpaces  infrastructure  supports  various  ways  of  managing,  controlling,  and  interacting 
with  the  shared  displays.  As  such,  individuals  can  interact  with  the  shared  displays  by 
simply  moving  the  mouse  cursor  from  their  own  screen  onto  the  shared  screens.  Users  can 
also  easily  share  the  information  on  their  own  screens  with  others.  The  LiveSpaces 
infrastructure  also  supports  the  federation  of  distributed  sets  of  LiveSpaces. 


Shared  displays  Automatic  lighting  Video  teleconferencing 


Re-configurable  table 


\  s 

Smart  whiteboards 


Personal  computers 


Figure  14  -  DSTO’s  Intense  Collaboration  Space ,  using  LiveSpaces  environment  ( Phillips ,  2008) 


9  Smart  Room  Environments 

Smart  room  environments  can  be  defined  as  physical  spaces  that  encourage  and  enable 
better  collaboration,  more  creative  thinking,  quicker  decision  making  and  increased 
productivity.  Command  centers  will  transform  into  smart  room  environments  meant  to 
support  analysts  and  watch  officers  in  their  work.  These  environments  will  be  equipped  with 
a  number  of  physical  devices  such  as:  large  group  displays,  high-end  analyst  workstations, 
multi-touch  tables,  digital  dash  boards  and  ambient  displays,  telepresence  and  video 
conferencing  equipment,  multiple  cameras  and  microphones,  room  and  people  sensors 
(motion  detectors,  acoustic  analyzer,  body  tracker,  eye-tracker),  room  controls  (software- 
controllable  lights  and  speakers),  biometric  authentication. 

From  a  user  experience  and  a  capability  perspective,  smart  room  environments  could 
provide:  ubiquitous  computing,  adaptive  user  interfaces,  speech  recognition  /  speech 
transcription,  scripted  audio-video  briefings,  shared  situation  awareness,  shared 
collaboration  awareness  (across  meeting  spaces),  and  supporting  (recording,  revisiting)  the 
collaborative  decision  process.  LiveSpaces  (Section  8.4)  is  a  good  example  of  a 
collaboration  infrastructure  to  support  such  a  capability. 


Naval  Space  and  Warfare  (SPAWAR)  Systems  Center  Pacific  in  San  Diego  has  developed  a 
prototype  of  a  facility  of  a  Command  Center  called  the  Navy  Command  Center  of  the  Future 
(CCoF).  It  is  equipped  with  glass  walls  with  many  small  video  screens;  a  large  video  screen 
that  serves  as  the  central  focus  of  those  in  the  room;  a  small,  and  a  private  sound-proof  room 
with  glass  walls  that  can  automatically  be  darkened  in  case  classified  information  needs  to 
be  discussed  (Figure  15).  On  both  sides  of  the  glass-walled  room,  several  video  displays  can 
be  used  to  show  live  video,  documents  loaded  by  an  artificial-intelligence  system,  news 
from  commercial  media,  and  just  about  anything  the  people  need  to  see.  Glass  surfaces  will 
allow  decision  makers  to  manipulate  data  and  other  information  simply  by  running  their 
fingers  over  the  glass.  CCoF  would  have  Artificial  Intelligence  (AI)  meant  to  discern  what  is 
being  talked  about  during  a  teleconference  and  to  know  how  to  source  up  whatever 
documents  are  needed  as  they’re  needed.  (Terdiman,  2009). 


Figure  15  -  Pictures  of  the  Navy  Command  Center  of  the  Future  (Terdiman,  2009) 


10  Mixed  Reality 

Mixed  reality  refers  to  the  area  between  the  two  extremes  of  the  real  world  and  a  fully 
virtual  world  (Figure  16).  In  mixed  reality  the  merging  of  real  and  virtual  worlds  produces 
new  environments  and  visualizations  where  physical  and  digital  objects  co-exist  in  real  time. 
Augmented  reality  and  mixed  reality  are  now  sometimes  used  as  synonyms  (Wikipedia, 
2011b). 


-  Mixed  Reality  (MR)  - 1 

◄ - ► 

Real  Augmented  Augmented  Virtual 

Environment  Reality  (AR)  Virtuality  (AV)  Environment 

Figure  16  -  Virtuality  Continuum  (Wikipedia,  2011b) 

“Virtual  reality  provides  a  computer-generated  3D  environment  that  surrounds  a  user  and 
responds  to  that  individual’s  actions  in  a  natural  way,  usually  through  immersive  head- 
mounted  displays  and  head  tracking.  Gloves  providing  hand  tracking  and  haptic  feedback 
may  be  used  as  well.  Room-based  systems  provide  a  3D  experience  for  multiple 
participants...  The  growing  popularity  of  3D  entertainment  using  3D  glasses  —  and 
ultimately  3D  screens  and  projections  that  do  not  require  glasses  —  may  relegate  immersive 
virtual  reality  to  permanently  niche  status”  (Gartner,  2010). 


Virtual  Reality  can  provide  value  for  intelligence  collators  by  providing  them  with  a 
platform  to  conduct  a  mission  rehearsal  in  the  environment  in  which  they  will  operate,  in 
particular  in  hostile  urban  settings. 

Augmented  Reality  or  mixed  reality  is  a  particular  technology  which  allows  adding 
sensations,  images  and  information  generated  by  a  computer  to  the  normally  perceived 
reality  (Figure  17).  Additional  information  on  the  real  environment  such  as  images,  writings 
and  virtual  objects  are  provided  to  the  user  through  special  visors  worn  by  the  individual  or 
projection  in  his  immediate  environment.  (Contactum,  2011) 

Augmented  reality  applications,  which  superimpose  information  on  the  user’s  view  of  the 
real  world,  rather  than  blocking  out  the  real  world,  will  overtake  virtual  reality  based  on 
rapidly  growing  activity  in  the  mobile  marketplace  (Gartner,  2010). 


Figure  17a  -  Army  Augmented  Reality  ( Contactum ,  2011); 
17b  -  Mirage  Augmented  Reality  system  ( Arcane ,  2011) 


The  Mirage  Augmented  Reality  System  (Figure  17b)  is  a  complete  solution  allowing  you  to 
create  your  own  Augmented  Reality  experience  by  inserting  virtual  content  to  the  real 
environment  and  using  stereo  video  see-through  OLED  HMD  (Arcane,  2011). 

11  Biometry 

Biometry  refers  to  the  automatic  identification  or  identity  verification  of  living  persons 
using  their  enduring  physical  or  behavioral  characteristics.  Among  the  features  measured 
are:  face,  fingerprints,  hand,  handwriting,  iris,  retinal,  vein,  Deoxyribonucleic  acid  (DNA) 
and  voice.  Biometric  technologies  are  becoming  the  foundation  of  an  extensive  array  of 
highly  secure  identification  and  personal  verification  solutions.  To  be  effective,  a  biometric 
system  must  compare  captured  biometric  data  to  a  biometric  database.  Emerging  biometric 
technologies  include:  DNA,  odor,  facial  thermography,  nail  bed,  skin  reflectivity  and  lip 
movement  recognitions  (GBT,  2011). 

In  a  military  context,  biometry  has  the  following  applicability: 

•  Authentication.  Security  management  of  users  to  a  facility  (e.g.  command  center), 
computer.  Security  management  of  the  privileges.  Profile  management  of  user  roles 
and  preferences. 

•  Identification  /  Recognition.  Identification  of  belligerents.  Forensics  operations  (e.g. 
scene  investigation  of  an  explosion). 


12  Augmented  Cognition 

Augmented  Cognition  (Aug  Cog)  is  an  emerging  field  of  science  that  has  the  explicit  goal  to 
extend  a  user’s  abilities  via  computational  technologies,  which  are  explicitly  designed  to 
address  bottlenecks,  limitations,  and  biases  in  cognition  and  to  improve  decision  making 
capabilities.  It  proposes  to  do  this  through  continual  background  sensing,  learning,  and 
inferences  to  understand  trends,  patterns,  and  situations  relevant  to  a  user’s  context  and 
goals  (ACIS,  2011). 

Augmentation  Cognition  has  significant  applicability  to  military  intelligence,  as  analysts 
must  frequently  perform  cognitively  demanding  tasks  in  stressful  environments.  Defense 
Advanced  Research  Program  Agency  (DARPA),  through  its  AugCog  Program  has 
developed  technologies  to  mitigate  sensory  or  cognitive  overload  and  restore  operational 
effectiveness  by  extending  the  information  management  capacity  of  the  warfighter.  This  is 
accomplished  via  closed-loop  computational  systems  strategies  including  (DARPA,  2011a): 

•  Intelligent  interruption  to  improve  limited  working  memory; 

•  Attention  management  to  improve  focus  during  complex  tasks; 

•  Cued  memory  retrieval  to  improve  situational  awareness  and  context  recovery; 

•  Modality  switching  (i.e.,  audio,  visual)  to  increase  information  throughput. 

13  Virtual  Assistant  /  Virtual  Advisor 

A  virtual  assistant  or  advisor  is  a  conversational,  computer-generated  character  that 
simulates  a  conversation  and  that  are  capable  of  providing  guidance,  bringing  updates, 
pointing  to  missing  elements,  and  generating  automatic  triggers  and  alerts  as  required.  A 
virtual  assistant  incorporates  natural-language  understanding,  dialogue  control,  domain 
knowledge  and  a  visual  appearance  that  changes  according  to  the  content  of  the  dialogue. 

In  a  military  intelligence  context,  virtual  assistants  will  be  growing  in  importance  in  terms  of 
providing  a  natural  and  contextual  interface  between  the  user  and  an  AI  engine  that  will 
perform  automated  reasoning  and  knowledge  management  activities  on  the  situation  and 
information  at  hand.  An  interesting  example  of  this  is  the  DARPA  initiative  Personalized 
Assistant  that  Learns  (DARPA,  201  lb). 

14  Advanced  HCI  Concepts  &  Techniques 

14.1  Advanced  information  visualization  /  Visual  Analytics 

“Visual  analytics  is  the  science  of  analytical  reasoning  facilitated  by  interactive  visual 
interfaces”  (Thomas  and  Cook,  2005).  “The  basic  idea  of  visual  analytics  is  to  combine  the 
strengths  of  automatic  data  analysis  with  the  visual  perception  and  analysis  capabilities  of 
the  human  user.  It  uses  visualizations,  user  interaction  and  data  analysis  techniques  to  find 
insight  from  complex,  conflicting  and  dynamic  information.  Visual  analytics  is  especially 
focused  on  situations  where  the  huge  amount  of  data  and  the  complexity  of  the  problem 
make  automatic  reasoning  impossible  without  human  interaction”  (Jarvinen  et  ah,  2009). 

Figure  18  provides  two  examples  of  Visual  Analytics.  The  first  one  shows  a  River  Theme 
where  the  main  news  feeds  are  depicted  in  time  and  allow  understanding  which  events  have 


drawn  more  attention  and  in  some  cases,  spun  off  other  topics  of  interest.  The  second 
example  is  a  Galaxy  view  based  on  the  extraction  of  concepts  from  a  large  set  of  documents 
and  the  regrouping  of  the  documents  into  concept  clusters. 

Visual  Analytics  is  going  through  a  significant  growth.  It  will  also  be  a  technology  of  choice 
in  the  military  intelligence  world  as  analysts  must  digest  vast  amounts  of  collected  data  and 
make  sense  of  them,  identify  patterns  and  trends.  In  particular,  through  the  use  of  Visual 
Analytics,  intelligence  analysts  will  be  able  to  (Thomas  and  Cook,  2005): 


•  Synthesize  information  and  derive  insight  from  massive,  dynamic,  ambiguous,  and 
often  conflicting  data; 

•  Detect  the  expected  and  discover  the  unexpected; 

•  Provide  timely,  defensible,  and  understandable  assessments;  and, 


Figure  18  -  Visual  Analytics  examples:  River  Theme  visualization  (Ribarsky,  2009); 
Galaxy  visualization  (Future  Point,  2011) 


14.2  Context  Sensitive  /  Adaptive  User  Interfaces 

An  adaptive  user  interface  changes  its  layout  and  element  of  information  based  on  the  user 
roles,  his  needs  and  preferences.  This  allows  tailoring  the  interface  to  the  task  at  hand. 
Adaptive  user  interfaces  are  also  good  to  overcome  the  information  overload  problem  by 
focussing  on  the  required  information  and  processes.  Eye-tracking  could  be  used  to  observe 
where  the  user  is  looking  at  and  customize  accordingly  the  user  interface. 


14.3  Advanced  Interface  Widgets 

A  widget  is  an  element  of  a  graphical  user  interface  that  displays  an  information 
arrangement  changeable  by  the  user,  such  as  a  window  or  a  text  box,  and  enabling  direct 
manipulation  of  a  given  kind  of  data  (Wikipedia,  2011a).  Interface  widgets  can  significantly 
facilitate  the  task  of  a  user.  Slap  widgets  (Figure  19)  are  physical  interface  components  that 
add  movable,  tactical  controls  to  a  device  such  as  a  multitouch  table. 


Figure  19  -  Slap  Widgets  (Weiss,  2009) 


14.4  Smart  Lenses 

As  illustrated  in  Figure  20,  smart  lenses  technology,  such  as  the  Pliable  Display  Technology 
from  Idelix  Systems,  allow  a  user  to  handle  vast  amounts  of  data  and  accurately  interact  with 
critical  details  in  context  (Baar  and  Shoemaker,  2004). 


Figure  20  -  Examples  of  Smart  Lenses  from  Idelix  (Baar  and  Shoemaker ,  2004) 


15  Precision  Information  Environments 

In  order  to  appreciate  how  the  evolution  of  HCI  could  benefit  future  intelligence  analysis,  it 
is  very  informative  to  examine  the  work  carried  out  by  the  US  Department  of  Homeland 
Security  (DHS)  and  Pacific  Northwest  National  Laboratory  (PNNL)  to  design  and  develop 
future  work  environments  for  the  emergency  management  community.  Called  Precision 
Information  Environments  (PIEs),  these  environments  are  meant  to  provide  planners  and 
responders  with  precise,  relevant  information  and  with  tools  that  aid  collaboration, 
information  sharing,  and  decision  support  (DHS  /  PNNL,  2010). 

A  video  illustrating  the  concepts  of  a  PIE  environment  to  support  fire  fighters  has  been 
developed  (DHS  /  PNNL,  2010)  and  exhibits  many  of  the  HCI  trends  described  in  the 
current  report.  Some  of  the  key  concepts  are  the  following: 

•  Pervasive  Sensor  Network.  Live  data  streams  from  instrumentation  in  the  field  give 
emergency  management  personnel  an  intimate  view  of  remote  conditions. 

•  Synthetic  Environments.  Data  feeds  from  the  field  are  synthesized  into  a  navigable 
virtual  environment  through  which  collaborators  can  better  understand  an  event  and 
explore  response  strategies. 

•  Adaptive  User  Interface.  User  models  define  the  roles,  responsibilities  and 
perspectives  of  each  person  using  a  PIE.  Software  interfaces  adapt  their  form  to 
support  each  user’s  tasks  and  preferences  (Figure  21a). 

•  Live  Status  Tracking.  The  tasks  of  every  PIE  user  are  tracked  so  that  a  clear  picture 
of  a  response  activity  is  always  available  (Figure  21b). 

•  Seamless  Information  Transfer.  Through  natural  interaction,  information  can  be 
moved  from  one  interface  to  another;  relevant  information  follows  the  user  and 
adapts  its  form  to  the  device  on  which  it  is  shown  (Figure  21c). 


•  Multimodal  Interaction.  Use  of  surface  computing  is  used  for  collaborative 
interaction  towards  shared  situation  awareness  and  decision  making  (Figure  2 Id). 


a:  Intelligent  contextual  desktop  b:Sm ait  dashboards 


■:  Gesture- based  drag-and-drop  d:  Surface- computing  collaboration 


Figure  21  -  Illustration  of  Precision  Information  Environments  concepts  (DHS  /  PNNL,  2010) 

Intuitive  Collaboration.  Distributed  users  can  interact  with  each  other  and  with  PIE 
software  as  if  they  were  co-located.  In  the  illustration  shown  in  Figure  22a,  distributed 
collaborators  interact  easily  with  each  other  and  with  the  information  using  a  live  wall. 
Remote-users  are  embedded  in  the  image,  in  mirror  mode  in  order  to  reflect  when  they  look 
at  the  same  information  on  live  wall. 

•  Advanced  HCI  Widgets.  Users  are  provided  with  advanced  graphical  user  interface 
widgets  and  natural  interaction  mechanisms  (Figure  22b). 

•  Seamless  Communications.  Information  that  is  relevant  to  the  user  given  his  role, 
mission  and  context  is  automatically  presented,  and  in  form  appropriate  to  the  device 
on  which  he  is  interacting. 

•  Ubiquitous  Displays.  The  PIE  architecture  allows  information  to  be  shared  across 
devices  easily  (Figure  22c);  to  support  more  in-depth  analysis,  the  view  from  a 
mobile  device  can  be  shared  with  a  nearby  display.  As  illustrated  in  Figure  22d,  the 
truck’s  windshield,  equipped  with  an  embedded  OLED  display,  is  used  as  an 
alternate  display  and  controlled  by  the  wrist  computer  as  an  input  device. 

•  Decision  Support  in  the  Field.  Personnel  in  the  field  will  have  as  much 
computational  support  as  those  in  the  operations  center.  By  understanding  each 
user’s  tasks,  PIE  offers  customized  data  and  recommendations. 

•  Augmented  Reality.  New  interfaces  will  give  users  a  unique  view  of  their  world, 
populating  it  with  context-sensitive,  task-appropriate  information. 


a:  Live  vval I  distributed  collaboration  b:  Advanced  HCI  'idgets 


c:  IV  earabl  e  di  spl  av's  d:  U  bi  quitous  com  puti  ng  /  novel  devi  -:es 


Figure  22  -  Illustration  of  Precision  Information  Environments  concepts  (DHS  /  PNNL,  2010) 

16  Conclusion 

The  nature  of  military  operations  has  changed  significantly  over  the  last  decade. 
Asymmetric  warfare,  urban  and  human  terrain  settings  have  increased  the  complexity  of 
these  operations.  Military  intelligence  analysts  are  faced  with  a  huge  amount  of  information, 
of  various  types,  which  they  need  to  make  sense  of,  in  order  to  improve  overall  situation 
awareness.  They  also  need  to  collaborate  with  different  military  and  civilian  organizations 
within  a  JIMP  context.  Luckily,  communication  and  information  technologies  have 
progressed  significantly  to  support  them  in  their  work. 

In  particular,  new  and  future  HCI  technology  provides  significant  opportunities  to  support 
deployed  intelligence  collators  as  well  as  analysts  working  in  command  centers.  The  trend  in 
ubiquitous  communications  and  computing  will  continue  and  people  will  be  able  to  interact 
with  the  technology  that  surrounds  them  in  more  accessible,  intuitive  and  less  restrictive 
ways.  Smart  phones  /  smart  personal  device  assistants  have  gained  a  significant  momentum 
and  will  be  provided  with  a  large  variety  of  advanced  intelligence  applications,  such  as 
augmented  reality  and  culturally-assisted  translation.  Smart  room  environments  will  provide 
better  shared  multi-intelligence  situation  awareness  and  collaboration.  Multimodal  and 
natural  interaction  will  be  the  common  interface  of  the  future.  Systems  will  be  designed  with 
intelligent,  adaptive  interfaces,  understanding  the  user  roles  and  the  situation  at  hand. 
Advanced  visual  analytics  approaches  will  allow  to  derive  insight,  identify  patterns  and 
trends.  Virtual  /  synthetic  environments  will  be  used  to  predict  situations. 

Although  this  paper  is  focussed  on  the  military  intelligence  capabilities,  it  would  be  of 
interest  to  broaden  the  scope  and  engage  in  a  multi-agency,  joint  and  coalition  discussion. 
The  HCI  trends  identified  in  this  paper  could  be  revisited  and  see  how  applicable  they  would 
be  to  modem  defence  and  security  operations,  considering  activities  such  as  humanitarian 
assistance  and  disaster  response  coordination. 
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Re-configurable  table 


N  / 

Smart  whiteboards 


Personal  computers 


Mixed  Reality  (MR) 


Mixed  Reality 


DEFENCE 


► 


Real  Augmented  Augmented  Virtual 

Environment  Reality  (AR)  Virtuality  (AV)  Environment 


Biometry 
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|  S3  GrFingerX  -  Sample  application  -  Visual 


Image  Options. . .  Version 


^JnJxJ 


“"GrFingerX  Initialized  Successful!"" 

Sensor:  File.  Event:  Plugged. 

Sensor:  MsfrO.  Event:  Plugged. 

Sensor:  MsfrO.  Event:  Finger  Placed. 

Sensor:  MsfrO.  Event:  Image  captured. 

T emplate  extracted  successfully.  High  quality. 
Sensor:  MsfrO.  Event:  Finger  removed. 


Enroll 

Verify 

Identify 

Extract  template 


|^£lea^databasej 

Auto  identify 
[7  Auto  Extract 


www.lisisoft.com 


www.dealspwn.com www.handresearch.com 


www.biometry.com 


Vein  pattern  palm  reading  by  Fujitsu 


Augmented  Cognition  /  Brain  Computer 
Interfaces 


Emotiv  EPOC 


http://web.cecs.pdx.edu/~stro 


NeuroSky  MindSet  OCZ  Technology  Neural 

Impulse  Controller 
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Virtual  Assistant  /  Virtual  Advisor 


DEFENCE 


IBM's  Watson  supercomputer 


DARPA’s  CALO 
Cognitive  Assistant 
that  Learns  and 
Organizes 


Pubmed  Faceoff 
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Advanced  HCI  Concepts  &  Techniques 


DEFENCE 


Smart  Lenses 


Visual  Analytics 


VAST(2008) 


18  Smart  Lenses  from  Idelix  (Baar  and  Shoemaker,  2004) 


Advanced  Interface  Widgets 


Slap  Widgets  (Weiss,  2009) 


Advanced  HCI  Concepts  &  Techniques 


www.precisioninformationenvironment.org 


Intelligent  /  contextual  desktops 


Smart  dashboards 


19  Gesture-based  drag-and-drop 


Surface  computing  collaboration 


Advanced  HCI  Concepts  &  Techniques 


www.precisioninformationenvironment.org 


20  Wearable  displays  Ubiquitous  Computing  /  Novel  Devices 


Conclusion 


DEFENCE 


•  Complex  military  operations 

•  Huge  amount  of  information  to  process 

•  Need  for  collaboration 

•  Continued  HCI  Trends 

-  Ubiquitous  communications  and  computing  will  expand 

-  Technology  more  accessible,  intuitive  and  less  restrictive 

-  Multimodal  and  natural  interaction,  the  common  interface  of  the  future 

-  Smart  phones  /  tablets  with  increasing  intelligent  applications 

-  Systems  will  be  designed  with  intelligent  and  adaptive  interfaces 

-  Smart  room  environments  to  enable  better  SA  and  collaboration 

-  Advanced  visual  analytics  to  derive  insight,  identify  patterns  and  trends 

-  Virtual  /  synthetic  environments  to  predict  situations 
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denis .  gouin@drdc-rddc .  gc.  ca 
valerie.lavigne@drdc-rddc.gc.ca 


